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ABSTRACT 

Data. collected and analyzed in this study over a 
three-year period lend credence to the notion that professional 
judgment is a reliable and stable criterion variable for teacher 
education research. Participants included 209 student teachers, 131 
classroom teachers, and 27 university teacher • trainers. Student 
teachers were ranked according to their teaching effectiveness by 
their supervising teachers, their university methods instructors and 
supervisors, their fellow student teachers , and by a self-rating 
procedure. Fourteen of fifteen correlations among the various 
effectiveness rankings were positive and statistically significant at 
the .01 level- Ihie remaining correlation was positive and significant 
at the .04 level. Student' teachers ranked as the icost effective 
professionally tended to be the most satisfied with their field-based 
teacher preparation program. Student teacher satisfaction with their 
training program was measured through use of the Purdue Student 
Teacher Opinionaire (PSTO^) . Supervising teachers expressed their 
perception of student teacher satisfaction by completing a Modified 
PSTO; There was considerable congruence between student teacher 
expressed satisfaction and teacher perception of tie degree of ' , 
student teacher sartisfaction. (Author/MM) 
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Professional Judgment as a Criterion Variable 
iri\Pre-Service Teacher Education Research 

James Mahan, .Indiana UuiVersity 
Jerome C. Harste, Indiana University 



Turner (1975) lists professional Judement as a criterion variable related to 
teacher work success. This study focuses upon the issue of longitudinal relia- 
bility, and stability of professional Judgment as' a criterion , variable. Specifi- 
cally, "the' purpose of the study vas to annually collect and interpret data pertinent 
to the folloving questions of central importance to evaluators of teacher e^^ducation 
outcomes : ' \ .. . 

(1) Do studeni teachers as a group, the individual student teacher, university 
methods and supervisory personnel, and supervising teachers agree in their 
professional judgments of the most effective student teachers? 

(2) Are student teacher evaluations of the student teaching experience, as 
measured by the Purdue Student Teacher Opinionaire (PSTO) and by a preparatory 
program evaluation instrument, related to the effectiveness rankings of student 
teachers by peers-, by. self , by xiniver'sity personnel, and by supervising 
teachers? 

t '. ^ . • ' , " 

(3) Are student teacher responses on selected subsets of the PSTO (i.e,. Rapport . 
vlth Students) related to one or more of the independent rankings of profes-' 
signal effectiveness? . . 

(U) . What degree of congruence is there between ■ student teacher evaluation of the 
student teaching ^experience via the PSTO and "supervising teacher evaluation of 
the same experience via a modified PSTO? 

(5) How stable are these relationships over three annuai .. replications? 
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RATIONALE- , " ■ 

V / 

Turner (1975) has suggested that the aiin, of research in teacher education is 
to optimize that portion of teacher work Buccess /attributable to teacher prepara- 
tion. Teacher work success is seen as a 'complex notion involving at Ifeast three 
classes of criterion variab&s: -{lO student iattainiaent, (2) professional judgment, 
and (3) student Judgment. In further explaaaation of . the criterion variabl^^ 
professional, judgment. Turner. states that the teacher's professional imege or 
stature, as judged by school and college supervisors, peers, and school administra- 
tors is an important dimension of teacher success. since virtually all teachers are 
employed in some kind of institutional setting or formal* work.* organization, 
'Included in a teacher's professional image may be ability to work with peers,, 
relation to parents and other commtmity members, perceived motives, knowledgeable- 
ness, leadership aualities, perceived skill, dependability, dedication and so on. 
One difficulty in optimizing teacher success is that the criteria mentioned by 
Turner above, and all of which are important, may not invariably' correlate posi-* V 
tively with each other. In certain instances inverse correlations may appear. 
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• Decreases in pupil dependency or euton6iny;may ai^conip'etfiy increafeeis in student 
acy.evement , /or high levels of studeitf "achiev^ent may be associated iilth low ^. ; 
ratings of instruction. If the ciji^erion, which represents teaplier worH succes^^ 
is indeed as Heterogeneous e.s suggested.eCbove, a distinct optimization problem Is ' ' 
posed. It/is difficult to determine the. means by which, and^thbse points up to ^ 
■ which, one: set of criterion variables can be increased without /inducing a decrease 
in ^another set of criterion variables . ,, Ppr example , to what point can student 
attainment "be increased without decreasing teacher- ability* to function in school 
work^ settings?. This same .difficulty may well exist within criterion variable ... 
cras^sesvfi^^ supervising teacher perceptions of teacher success may well vary : 
' fj^oii;- Wiv^ personfiel perceptions of the phenomenon, and both may vary from 

peer 'percept ions . 

, Given;: these possibilities, the purpose, of this study is seen as one designed 
^^^s to exploiris the. nature of the .relationsiaips which > exist -within the criterion' vari- 
able of 'professional " Judgment . Conflicting qualitative; perceptions, of teacher ^ 
performance are olten ass'tmed to be an issue confronting teacher education evetlu- 
ators. This study provides needed .data relative to that issue. . 

' . ■ / ' ■ •■• • \: ■ ■ : • -'^ ' ■ . • ■ 

PROCiaDroES ^TO. INSTR.UIffi'OTATION 

Four independent rankings of student „teacher_.effectiveness were obtained at 
. \ the .end of the 19:72-73, 197^3-7^, .and 197^75 academic years. .Data were obtained 
:f^om ail' participants .invol\red .i field-based teacher edudation 

program and included .U$ classroom teachers, 85 student teachers, and 10 xmiversity 
methods instructors and supervisory specialists in 1972-73; ^5 classroom teachers, 
7Q student teachers, and .8 university personnel in 1973-7^ j. and hO classroom • . \ 
■ ,l teachers, 5i+ -student teachers i a^ in 197^-75. 

Program. Satisfaction Measures ^ Student teacher satisfaction with the student 
teaching program was dete:rmined^through administration of .two different instruments, 
the Purdue Student Teacher Opinionaire (FSTO) and a questionnaire developed tb 
provide specific inforaaation about key dimensions of the field-based training prp- 
gram., the Program Evaluation Scale (PiSS).'. 'The PSTO has high validity and relia- 
bility coefficients (Bentley and Price, I969) and. has been employed frequently to 
measure satisfaction with student .teaching- conditions in bot^^ 
" innovative programs. The PES has been ^employed is.uccessfully since .1973 'iii the .> 
evaluation of the ri^ferent teacher training j)ro^ram ..(Mah^ . 
A Modified PSTO, 'revised to measure teacher Vs, own opinions of the 

si:udent teacher's satisfaction with the conditions and relationships- encountered- . 
dtiring student teaching, ^ma adminiat.erisd to each supervising teacher. By using 
the. Modified PSTO, a comparison of student teacher satisfaction with- the classroom 
. ,^ teacher's perception of that satisfaction* was pdssilDle, i.e>, where the student 
i ' teacher reacted to a statement like,, "relations yith other teachers in the 
v/ " * ' building were. . . /V the classroom teacher reacted to the reworded statement ^ "The 
relations of iny student teacher with other teachers in the building were \ 

■ .student teachex*^ made a second e of satisfaction with liiheir training 

::| " program via. >the Program Evaluation Scale ^^^^ -' A: ten; point continiim ranging from " 

. ^'I'V (no contribution) to. ^'10'^ (grea* qontrib^io^ ^sess th^ • r 

degree- to which the student te? liters felt that eight different aspecl^s of the pi;;o- .; 
V gram contributed -to tlieir professional development. The aspects were i, ^ . ; V ' 
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supervisory assistance froni classroom teachers , 

- instructional preparation received from methods professors 
assistance received from supervision specialists 

- supervisory assistance received from methods prof eissors • - 

- .advantaf?;es of "being placed- in 2 or more schools ' ^ 

- advantages of working vrith 2 or more groups of pupils 

. - assistance received from peers as a result of the '^clust^" approach 
to student teaching ' . / \ . 

- preparatory impact of the total program ■ ^' 

A student teacher could record sumnated evaluative scores as lev as 8 or as ' 
hsfgh.as 80 on the PES and as 1o\t as 100. or as high\as UOO on the.PSTO. Those vith . 
■the highest PES scores and highest PSTO scores were considered to be the most' sat- 
isfied/ with their year-lpng student teaching program as measured by the specific 
instrument. A highest third, middle third, and "bottom third of student teachers . 
was identified on the basis of summated PES scores and aj^ain* on the/basis of the 
suramated PSTO scores. 

Student Teacher Effectiveness Measures ,. Professional. effectiveness rankings 
collected included (l) student teacher ^rankings of peers, (2). classroom teacher 
rankings of student' teachers , (3) university staff rankings of student teachers, 
and (t) student teacher self ranlcings, ... 

Teacher e.f fectiveness was defined by each group operating independently but / 
with the input of the other professional groups involved^ in the study. Tliis was/ 
accomi)lished in the following manner ; ■ . . / 

(l) A whole group meeting involving; classroom teachers, student teachers/ and 
university methods persons was held to discuss various criteria for/ 
judging- teacher effectiveness.,,.: All participa.nts understood that teacher 
effectiveness was a complex notion oftentimes defying eaisy explanation 
and that, therefore, each group , could independently select the filial 

crite^ria they .would use • " . / ^ / 

■ ■ • . / ■ 

• <? f •■ 

■ (2) Following this meeting, each group met independently to. arrive qn ,the 
criteria it- would employ in Judging teacher effectiveness. 

Using the .procedural preferences expressed by student teachers relative. to / 
the Student Teacher Ranking by; Peers, the. names of all student teachers/ were 
placed on a sheet of pamper. This listing was given to the student teacliers along 
with instructions to rank order the top six sttident teachers with" respect to their 
^classroom teaching effectiveness. All responses were anonymous. The/ sum of ranks 
for each student teacher, based upon. the responses of peers/ was' computed. Student 
teachers w^th the highest sum. of ranks were considered to be the mo s^ effect j:Ve 
professionally. . Those student. teachers receiving few or no votes "^pre consiciered/ 
to be least effe.ctive professionally. Tlie. student teachers were t^en divided into 
a highest , third, middle third, and bottom third on a basis of professional /effec;^ 
•tiveness for data analysis purposes. / / 



- -.i^-^- which :were expressed by classroom teacher^/ 

Ined by having' each classroom, teacher rate - 



Using the^ procedural preferences 



rankings of student teachers, were obtained by having 

each of "his/her Vinal two student teachers on 10 criteria for ^/^prof essional ;ef fee 
tiveness. A five\ point scale ranging- from ^'l"' (some) to 5" . /(out standing) was 
employed. The criteria were: 



\ . 

\ 
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- effective classroom" itianageraent 

- creation of -a motivating learning environment - 

- preparation of sound lesson plans :. 

r ver'bal reinforcement of pupils ' ■ i • . 

sensitivity to individual needs, of pupil • - 

- relia'bility " ■" i ; 

- acceptance of classroom responsibility^^- '. ' ,\ 

- readiness for daily teaching duties . . ^ 

- enthusiasm for teaching , " 

- mastery of general teaching skills " * 

' ' ■ ■ ■ ... p 

Each student 'teacher, could earn a summated score of from 10 to 50. Student teachers 
. vlth the highest total score ^pn the 10 'criteria were considered to be the most 
effective professionally. The studetit teachers were then divided -into a higlaest 
third, middle third, and bottom' third for data analysis purposes. 

University Staff Rankings of Student Teachers vere obtained by having ea^ch 
staff member rate every student teacher on five criteria. A five point scale, 
ranging from (unsatisfactory) to "5" (outstanding) i-ms employed. The ca:-iteria 
were: - rapport \n.th children, planning for 'instruction, initiative and creativity , 
classroom management, and implementation 'of teaching' plans and activities^ Class- 
room teachers and staff members in a series of conferences concerning the evalu- 
ation of student teachers had agreed that th^se five criteria*- included the ten * 
criteria utilized by the classroom teachers. * Student, teachers .with the highest 
summated score *on the five criteria vere considered to be the most effective pro-^ 
fessionaily. Tliose ifith the lovest scoires vere considered to be the least effective 
professionally. For data analysis purposes, the student teachers were again 
divided into a highest, middle, and bottom third.' . ' 

*' Student Teacher Self Rankings were obtained "by having each student teacher* 
rate his or her self on the same ten . criteria Used by. the 'Classroom teacher in 
rating the student teachers. The same five point scale was employed. Each. student 
teacher^ could award himself from 10 to .50 points. Student teachers receiving the 
most points from themselves, were considered the. most effective, professionally. / ;\. 
Classification into .a highest, middle, and bottom third was done. ' 

Data Ana lysis Procedures . Spearman's Coefficient of Rank Correlation (Siegel, 
-1956)"was. used to exaraine .relationships among the four rankings of student teacher 
effectiveness, .the\PSTb,..and the PES. " Similarities ^and differences between pre- 
service and in-service teacher responses on the 'PSTO, Modified PSTO^, r.nd selected* 
•subscales of the PSTO vrere examined through use of arithmetic means and graphs. 



- ■■ RESULTS. .MDDISCUSSIOIJ ... 

•Foi^r-tsen of -^h^' fifteen correlations displayed in Table I are positive and 
^statistically significant at the .01 level. 'Furthermore, these rank-border corre- 
lations indicate'a tendency of the underlying ranks to relate in a mo no tone- 
increasing manner .(Hays, 19''^3, p. ^53) . For example^ low rankings .by student 
teache.rS/ are associated vrith correspondingly .Ibv. ranks., by supervising teachers ; 
and. so- .oni'- ■ T^ significant correlatioris>;,.occiurring over three successive years 
in an on-going, field-based teacher prepr?.ration program suggest considerable 
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SpeaoiTaan Rank-Oraer Correlations for Four Rankings 
of Student Teacher Effectiveness 
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*No Student Teacher Splf Ratings vere obtained at the end of the 1972-73 
academic year. '"" 



agreeinent among individual student teachers, student teachefs as a group, diversity 
personnel, and classroom teachers concerning vho is, and who is not, a profession- 
ally effect iye Gtudent teacher. Table I data indicate that .the various program .. 
participants tended to select about the same subgroup 'of students as ''most effective 
and the same, subgroup as "least effective."* These perceptions of effectiveness vere 
stable from year-to-year.. One explanation for this findifig^in this program is that 
university personnel and classroom teachers probably had /reetched many common educa- 
tional expectations and operational agreements in the th:^ee years this program 
existed prior 'to this study. . These common expectations,itransmitted to student 
teachers at the beginning. of the academic yea^., ho dpubt i led to the. various types 
of program -^participants sharing similar , perceptions of teaching success. If this 
explanation is accepted then . one might expect that if a similar study had. been con- 
ducted in the first' year of implementation- of the same plfeparation program it might 
.have revealed more. conflicting views of . teaching successV 

An aiteraa.te explanation bf the data id that, professional judgment of teacher 
^effectiveness .is a .fclativelyi^stable and reliable criteridri variable despite the 
profession' s seedling inability to operationally define ' it . - Many of us in f ield-^ 
base.d .programs have alxmys suspected^ this , feeling intuitive judgments about teacher 
effectiveness were ;better than the performance profiles Iwhich our data^ provided. . 



Our own experience operating field-based programs allovs ua to concliuie that almost 
everyone can agree- informally on who are the really *'good" teachers and ,vho are^ the 
really "weak" ones., The. problem. is not agreement, hut definition of the items among 
which ■ agreement is found. ' •» 

i • . • • 

^ TABLE II ^ - ■ • 

c " . . . . . • '. 

/ ' Spearman Rank-Order Correlations Between -Four Rankings 'of * Student Teacher 
^ Effectiveness and *"Satisfactidn With the Student Teaching Experience • ' 

as Measuredchy the PSTO and PES 
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: .^♦The PES device and the Student Teacher, Self Rating Device were nOt 
utilized at the. end of the 1972-73 academic year^ vt ,i 



Table .II indicates vhat student teachers who are perceived as • highly ei'f act iv^^ 
professionally in a fieiLd-based preparation program tend. to perceive the'?program 
itself . as highly .effective^ "when they are evaluating the program using:tjbii^^^PESr 
scale, an instrument designed to specifically address the. major characteristics 
of the program. Relative to. the- standardized -PSTO scale (where . some subsqales 
deal with forces "beyond university influence like -school cizrricultmi andVm 
community support, and principal' s leadership) -there is, due to the 1972^:-73 ^^"^^^ 
a.less stable tendency fpr the ""best student teachers" to be the "most ;satisfie^^ 
with their year--iong-. student /teaching experi . No clear- pattern emerged, between 
a student teacher *s satisfaction with the -student teaching experience as^Measured 
by the PSTO. and. the. effectiveness .ranking attributed to that .student teacher by 
peers. . One explanation, for the negative correlations between PSTO satisfaction and' 
rankings in xs that in years of teacher unrest or budget reduction' the i, 



compdnents of the PSTO that focuff upon coiamxmity, administration, and puhlic school 
curriculum can evoke negative responses f rom 1:he "best" student teachers. Effec-. 
ttve teachers may, for example, be the, most sensitive to breakdowns in either the 
community's or the university's prpyision of quality educational services and may 
assign low evaluations the responsible' party. Turbulence characterized the_ 
1972-73 year for ?ai-ticiyants in this study. School board- decisions to form middle 
schools and to change- attendance areas, reassign elementary pupils, and transfer 
teachers upset both supervising teachers and student teachers. Negative correla- 
tions between professional effectiveness and satisfaction with the program as 
measured by the PSTO occur only in 1972-73. It is vmfortunate that no. PES data was 
collected in 1972-73. If PES data had been obtained, and if it had correlated r~ 
positively mth effectiveness rankings, the argument that community variables 
beyond university staff control strongly eff-cted student teacher satisfaction- 
would be stfengthensd. . : - . ' ^ . 



TABLE III 



Spearman Rank-Order Correlations Between Scores on Selected Subsets 
of the PSTO and the Four Rankings of .Student Teacher Effectiveness 
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- Table III reveals that, students 'who thought most highly of their own professional 
: effectiveness, as measured hy PSTO, were also the most . satisf:^ed with teaching as ; 

■ ' a p«:ofessiop, with their relationships with the university supernrision s 

Bsxd with the . quality of methods /Instruction (professional preparation) -receive^^^^ 
- from the methods professors. The same students were also rather positiye about 
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. their rapport vith- their alem^ school pupil si * It is also evi dent v that satlW 

faction vith professional preparation .(positive feelings ahout methods instruction) 
was positively .associated, with high effectiveness ratings, Althoxigh correlations 
fluctuate^ from year-rtdryear, the general' trend" is for the, student teachers judged 
most effective to hey^'the student teachers most satisfied with, the five selected 

/.aspects of their, teacher preparation program as measured by the PSTO. ' 

Graph I presents a comparison het^/reett student teacher satisfaction with student 
teaching as measured "by the PSTO and. -supervising teacher perception of that student • 
teacher satisfaction as measured by a Modified' PSTO, , • 

Subsoales of these, instruments are:. • • - ■ . " - 

• * ■ -ir, • ■ , .. - . 

Factor 1 . Rapport with Supervising Teacher ! ■ ' • ' . 

Factor 2 . Rapport with Principal ^ ^ 

Factors. Teaphibg as a Profes'sion t:^ 

Factor k . Rapport with University Supervisor " - 

^ Factor 5 . Community Stxpport..of Education . * 

Factor 6 .' Student Teacher I^oad 

Factor 7 , Rapport, with Students \ ' . . 

Factor 8 . Rapport. -ivith Other Teachers 

Factor 9 «. Satisfaction with Housing 

Factor TO . . Professional" Preparation ' ' - 

Factor 11 , School Facilli ties , • ■ . 

Factor 12 . Curriculum' Issues • . - 

■ ^ - ' * ■ ■ In * ...... - - ■ ■ 

■ ^ ^ 

If .one defines .5 (one-sixth of the; range) as a difference of practical concern , 
there are few differences omong group means that"* are • significant . ^Generally;' class- 
room teachers . rather accurately perceived, the extent of student teacher "satisfact.ion- 
with the . year-long field experience. ' ' ' , " / 

■ The 197^-19'^5 teachers an^ student teachers had significant differences regar 
ing Factor 2 > Rapport^ with Principal,, and Factor 8 ^ Rapport with Other Teachers i . • 
The 1972-1973 student* teacher/supervising teacher groups reveled differences of ,5 
or greater. on two factors, also;- Factor 5 ^ Community Support "^F Education, and 
Factor 8, Rapport with Other Teachers. The classroom teachers in these cases - 
thought satisfaction was greater than it actually was. The disparity between - stu- 
dent teacher and classroom teacher on the issue of satisfaction" with coiomunity/ 
support, to some degree lends credence to the earlier? interpretation given to the 
negative correlations displayed in Table II for 1972-73. - * 
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• Then too, there seems to be a consistent -tjendency for cla^^room teachers to ^ 
feel . student' teachers 6ave better rapport with the remainder of the faculty than * . 
actually exists. This phenomenon occurred in both years « One explanation for this 
finding is that -Student, teachers probably do not share' vitb. the cvirrent supervising 
teacher^ all the -delicate 'details : of a ■ strained relationship • vith a dif f erent teach^ 
. elsewhere' dn the building;; Another explanati^^ is th?it some superyisiiig, teachers 
vho may be virtual ''loners" in ^'terms of school socializing may not • even imagine 
that a student teacher desires open and friendly iiteraiction with many classroom' 
teachers not just one. Furthermore, almost all the classr$p2js in this program 
.were '."self •-contained;"; It" may be that inservice" teachers ^pe forgetting the limita- 
tions the self-contained classroom places, on intra- faciilit^ communication, .grade--to- 
grade shari-ng, team teaching,! or even on a .xsbiiimpn coffee* break -time/ idealistic 
beginners on the other hand, desire and. ant ^ipate,. these close collaborative condi- 
tions and are -dilsappointed if they do not exist./- 
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- A' thioid and final explanation for the variation, in;the m6an scores between 
groups on Factor 5 anov Factor 8 is th^ these subscores are the least stable of all 
scores on, the PSTO. This explanation has 'some basis in that an excunination of the .. 
PSTO reveals these subscales are made up of only.*5 and 6 items respectively. While 
th4 least exciting, this explanation my well be the most plausible. If acciepted, 
GrapJOL^l*oyides even* stronger evidence supporting the realiability and s,t ability 
hypothesis being tested in ^his paper. " • • x,.. • 
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CONCLUSIONS . 



>■ . ,Data collected and ar!^y2ed in thi'fe "fetud^;" over a three year period lend', cre- 
dence to ^they notion that professional judgment is. a reliable criterion variable ;f or 
teacher education r exarch. Whether or not professional.," Judgment is as reliable; as 
. student attaiment arid student Judgment is an ezupirical issue whidh* needs , to be - . 
studied. .^Jjnilarly, whether* or 4io$ professional Judgment is a voilid measure of 
teacher ^wbrlc.Nsuccess., or whether or , riot prof essiopal -judgment is as valid 6. measure • 
• as- are student Judgment and .^tudent attainments were issues not researched, and - 
.^clearJ.y, ought ' to be* Asking questions of this sort; ma^r, however,, be .misleading, 
: ,for' somehow the mere act of suggesting that tl;Le validity of professional Judgment 
be studied elevates^, a,nd with no clear support , the! stature, of student attainment 
and. student^ Judgment as valid criterion" variables ,for Judging teacher work. success. 
At this point'there is .little to suggest -that- one criterion variabie, is more valid . 
than another^ .If anything, the^ record* -favors prof ess ional^. Judgment . .The long timS* 
favorite, student attainment, must ^ cert ainlj* be \g.uest 
: problems identified by ^oar *and Soar. (1975) • Then- too,, atu^^ logically 
suffers-in- thafc it .is a product .measture land teaching is,"***©^?^!!, agree i a-^rather ^com- 
plex and dynamic progess . ^Pr^^feasional Judgment as a criterion -variable both * - 
f pe^Tiiits ' and allows^ this importOT^t^. dimension of teaching| to enter .^into thfe evaltiation 
■pipcefes:?^Ahd this' we believe^ Is" a very positive arid desirable: fdature of^^ 
siohal Judgment as^ a criterion^ variable- in teacher e.ducatlon- research-. This pointv • 
however, probiibly^ is .that - • ^.>' 

teaching is fe, cbmplex arid dynamic process, and whiie l*$^arciers^ in* the . field wo^^ 
'/readily agree^^^their most ' comoniy^ 'employed B^thods and criterion variables* treat .it 
• • •• .as ■ skills andlas a product re^I)er^than a process. . Corisiderv^' for *^ 
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example, the vrealth of research on sliill or conpetencjr-based teacher education. 
>/hile there exists a reasonahlei^-ainoiint of information on problem identification and 
task-structtiring,. little. i$ known a"bout the fiatiire of lis. become inte- 

grated into the process of teaching and into the be'^ toire as demon- 

strated in 'a classroom event. Given such limit at ic :e£> "ol judgment does 

at least permit a rich- mix of va^riables to operate ^ . j^jlri^ che process of teach- 
ing and, at this point, may yell be our most valid ana i^ellable measure of effective- 
ness. At any rate, the data here, is encouraging and does suggest a high de^ 
•.stability in judgment across years .^eveh when one group (students) changed its entire 
membership each year. Although w#'can only ^speciaate, the data does suggest some 
^rather consistent definitions of teacher- effectiveness wer'e being applied across 
groups by the. evaluators,. It would not seem surprising, therefore, if later . ^ 
researchers fbund that professional judgment as a criterion variable 'captured^ some 
dynamic dimensions of the teaching process" which were illusive or left unaddressed 
im current outcome or student attainment studies. Tfte agenda ahead, and clearly 
no\sm.all one, is for us to .learn to • trust the criterioH^ variable of professional . 
Judgment. ■ • • ' ■ ' ' \ 
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